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Abstract. The classical theorem of Weitzcnbock states that the algebra of 
invariants K[X]^ of a single unipotent transformation g G GLm{K) acting on 
the polynomial algebra A'[X] = K[x\, . . . ,Xm] over a field K of characteristic 
is finitely generated. This algebra coincides with the algebra of constants 
of a linear locally nilpotent derivation S of ii'lX]. Recently the author 
and C. K. Gupta have started the study of the algebra of invariants i^m(2J)® 
where is the relatively free algebra of rank m in a variety 2J of asso- 

ciative algebras. They have shown that Fm{^)^ is not finitely generated if QJ 
contains the algebra UT2{K) of 2 X 2 upper triangular matrices. The main re- 
sult of the present paper is that the algebra Fm{^)^ is finitely generated if and 
only if the variety 2J does not contain the algebra UT2{K). As a by-product 
of the proof we have established also the finite generation of the algebra of 
invariants where Tnm is the mixed trace algebra generated by m generic 

n X n matrices and the traces of their products. 



Let K be any field of characteristic and let X = {xi, . . . , Xm}, where m > 1. 
Let g G GLrn = GLm (K) be a unipotent linear operator acting on the vector space 
KX = Kxi © • ■ -(BKxjn- By the classical theorem of Weitzenbock ^2], the algebra 
of invariants 



is finitely generated. A proof in modern language was given by Seshadri An 
elementary proof based on the ideas of ^21 was presented by Tyc . Since g — 1 
is a nilpotent linear operator of KX^ we may consider the linear locally nilpotent 
derivation 



called a Weitzenbock derivation. (The X-linear operator 6 acting on an algebra A 
is called a derivation if 6{uv) — S{u)v + u5{v) for all u,v G A.) The algebra of 
invariants C[X]^ coincides with the algebra of constants C[X]'' {— ker((5) ). See the 
book by Nowicki _10 for a background on the properties of the algebras of constants 
of Weitzenbock derivations. 

Looking for noncommutative generalizations of invariant theory, see e. g. the 
survey by Formanek 8 , let K{X) = K{xi, . . . , Xm) be the free unitary associative 
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algebra freely generated by X. The action of GLm is extended diagonally on K{X) 
by the rule 

For any Pl-algebra R, let T{R) C K{X) be the T-ideal of all polynomial identities 
in m variables satisfied by R. The class 03 — var(i?) of all algebras satisfying the 
identities of R is called the variety of algebras generated by R (or determined by 
the polynomial identities of R). The factor algebra Fm(Q3) = K{X)/T{R) is called 
the relatively free algebra of rank m in 03. We shall use the same symbols xj and 
X for the generators of Fm{'^)- Since T{R) is GL^-invariant, the action of GL,n 
on K{X) is inherited by Fm(03) and one can consider the algebra of invariants 
Fm{^)'^ for any linear group G. As in the commutative case, if 5 G GL^ is 
unipotent, then Fm(03)^ coincides with the algebra Fm{^Y of the constants of the 
derivation 5 — logy. 

Till the end of the paper we fix the integer m > 1, the variety 03, the unipotent 
linear operator g G GLm and the derivation 5 = \ogg. 

The author and C. K. Gupta [S] have started the study of the algebra of invariants 
Fm{^Y ■ They have shown that if 03 contains the algebra UT2{K) of 2 x 2 upper 
triangular matrices and g is different from the identity of GL^, then Fm(03)^ is not 
finitely generated for any m > 1. They have also established that, if UT2{K) does 
not belong to 03, then, for m — 2, the algebra ^2(2?)^ is finitely generated. 

In the present paper we close the problem for which varieties 03 and which m 
the algebra Fm(03)^ is finitely generated. Our main result is that this holds, and 
for all TO > 1, if and only if the variety 03 does not contain the algebra UT2{K). 

It is natural to expect such a result by two reasons. First, it follows from the 
proof of Tyc see also the earlier paper by Onoda ^I], that the algebra iir[X]f 
is isomorphic to the algebra of invariants of certain S'L2-action on the polyno- 
mial algebra in to + 2 variables. One can prove a similar fact for i^m(23)^ and 
{K[yi,y2] ®K Fm{^))^^^ ■ Second, the results of Vonessen ^B], Domokos and the 
author [H] give that i^m(03)'^ is finitely generated for all reductive G if and only if 
the finitely generated algebras in 03 are one-side noetherian. For unitary algebras 
this means that 03 does not contain UT2{K) or, equaivalently, 03 satisfies the Engel 
identity [x2,xi,. . . ,xi] = 0. In our proof we use the so called proper polynomial 
identities introduced by Specht the fact that the Engel identity implies that the 
vector space of proper polynomials in i^j„(53) is finite dimensional and hence F„j(03) 
has a series of ideals such that the factors are finitely generated iir[X]-modules. As 
a by-product of the proof we have established also the finite generation of the alge- 
bra of invariants T^„, where Tnm is the mixed trace algebra generated by to generic 
nxn matrices xi, . . . , Xm and and the traces of their products tr^Xi-^^ ■ ■ ■ Xi^), fc > 1. 

1. Preliminaries 

We fix two finite dimensional vector spaces U and V ^ dim U ~ p, dim V ^ q, 
and representations of the infinite cyclic group G = (g): 

pu ■■G~> GL{U) = GLp, pv -G-^ GL{V) = GLq, 

where pu{g) and pvig) are unipotent linear operators. Fixing bases Y — {j/i, . . . , yp} 
and Z — {zi, . . . , Zq} of U and V, respectively, we consider the free left iir[y]-module 



INVARIANTS OF UNIPOTENT TRANSFORMATIONS 



3 



M{Y, Z) with basis Z. Then g acts diagonally on M{Y, Z) by the rule 

9-^fj{yi,---,yp)zj ->^fj{giyi),...,giyp))gizj), fj £ K[Y], 

where, by definition, g{yi) = pu{g){Vi) and g{zj) = pv{g){zj). Let M(y, Z)^ be 
the set of fixed points in M(Y,Z) under the action of g. Since pu{g) and pv{g) 
are unipotent operators, the operators 5u = log pu{g) and 5v = loEPvig) are well 
defined. Denote by S the induced derivation of -^^[F]. We extend (5 to a derivation 
of M{Y, Z), denoted also by S, i. e. S is the linear operator of M{Y, Z) defined by 

i=i j=i j=i 

It is easy to see that 5 — log 5 on M(y, Z) and M (F, Zy coincides with the kernel 
of (5, i. e. the set of constants M (F, ZY . Changing the bases of U and V , we may 
assume that 5u and 5v have the form 
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where is the (r + 1) x (r + 1) Jordan cell 
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with zero diagonal. 

We denote by Wr the irreducible (r + l)-dimensional S'L2-module. It is isomor- 
phic to the 5'L2-iiiodule of the forms of degree r in two variables x,y. This is the 
unique structure of an 5'L2-module on the (r + l)-dimensional vector space which 
agrees with the action of S (and hence of g) as the Jordan cell (Q: We can think of 
S as the derivation of K[x, y] defined by 6{x) = 0, d{y) = x. We fix the "canonical" 
basis of Wr 



(2) 



= x'' 



,(1) = 



Ar-1) _ 



xy' 



1! ' 2! '■■ 

We give U and V the structure of S'i2-niodules 

C/ - T4^„, © • • • ® iy„. , V^W, 



(r-1)!' 



r\ 



(3) u — vvp^ • • • TO '^Pki " — qi 

and extend it on K\Y] and M{Y^Z) via the diagonal action of SL2- Again, this 
agrees with the action of g and 5. Then K\U] and M{Y.Z) are direct sums of 
irreducible S'i2-modules Uri C K\Y] and Wrj C M{Y, Z) isomorphic to Wr, i,j 



1,2,. ..,r = 0,1,2, 
respectively. 



, with canonical bases {u':^^ , u^]^ , 



} and {w'f-' ,wl^^ , 
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Lemma 1. The elements u e K\Y] and w G M{Y,Z) belong to K[Y]^ and 
M{Y, Zy , respectively, if and only if they have the form 

(4) " = X] '^"'^'^ri ' ^ = X! l^-^o^ri' ' ^ ^■ 

r,i r,j 

Proof. Almkvist, Dicks and Formanek ^ translated in the language of g-invariants 
results of De Concini, Eisenbud and Procesi |2] and proved that, in our notation, 
g{u) = u and g{w) — w ii and only if u and w have the form Q. Since g{u) = u if 
and only if 5{u) = 0, and similarly for w, we obtain that I^J holds if and only if u 
and w are (5-constants. (The same fact is contained in the paper by Tyc 1141 but in 
the language of representations of the Lie algebra sl2{K).) □ 

In each component Wr of t/ in ((SJ, using the basis we define a linear operator 
dhy 

d{u^^^) = {k + l){r - k)u^^+^\ fc = 0,l,2,...,r, 

i. e., up to multiplicative constants, d acts by u^"-* — > u'^'^^ u'^^-' • • • ^ 0. 
We extend d to a derivation of As in the case of (5, again we can think of d 

as the derivation of K[x, y\ defined by d{x) = y, d{y) = 0. 

Lemma 2. (i) The derivation d acts on each irreducible component Uri of K\Y] 
by 

diu-^^) = [k + l){r - k)u^^^\ fc = 0,l,...,r. 

(ii) If f ^ f{Y) E K[Y], then S''^^{f) ^ if and only if f belongs to the vector 
space 

s 

(5) K[Y]^=Y,d\K[Y]'). 

t=a 

Proof. Part (i) follows from the fact that the S'L2-action on U is the only action 
which agrees with the action of 6 as well as with the action of d (as the derivations 
of K[x,y] defined by 6{x) = 0, 6{y) = x and d{x) = y, d{y) = 0, respectively), and 
the extension of this 5'L2-action to K[U] also agrees with the action of 5 and d on 
K[U]. 

(ii) Since the irreducible 5'L2-submodules of K\Y] are 5- and d-invariant, it is 
sufhcient to prove the statement only for / G Wr C Considering the basis 

P of Wr, we have that ^^+H/) if and only if 

/ = oqw^"^ + "1"*^' H l-asW^^\ ak € K. 

Since = Ku^°^ and d\u^°^) e Ku^*\ we obtain that Wr n K[Y]s is spanned by 
u^^\ . . . , M^*^ and coincides with the kernel of 5^*+^ in Wr. □ 

In principle, the proof of the following proposition can be obtained following the 
main steps of the proof of Tyc jl4 of the Weitzenbock theorem. The proof of the 
three main lemmas in uses only the fact that the ideals of the algebra K[Y] 
are finitely generated ^/^[yj-modules. Instead, we shall give a direct proof, using 
the idea of the proof of Lemma 3 in 14 . 

Proposition 3. The set of constants MiY.ZY is a finitely generated K\YY - 
module. 
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Proof. Let Ni be the i4r[y]-submodule of M{Y, Z) generated by the basis elements 
Zj oiV = Kzi® - ■ -(BKzq corresponding to the i-th Jordan cell Jq^ . Since M(y, Z) = 
iVi © • • • © iV; and M{Y,Zy = A^f © • • • © iV/, it is sufficient to show that each 
Nf is a finitely generated i4r[F] ''-module. Hence, without loss of generality we may 
assume that q = r + I and S{zo) = 0, 6{zj) = Zj-i, j = f , 2, . . . , r. Let 

(6) / = /o(y)zo + /i(r)zi + ••• + /, (r)z,,eAf(r,z)^ /,(r)ei^[y]. 

Then 

Sif) = (Sifo) + fl)zo + + .f2)zi + ■■■ + {S{fr-l) + fr)Zr-l + 5{fr)Zr 

and this implies that 

5{f,) = -f,+i, j=0,l,...,r-l, 

5{fr) = 6^{fr-i) - • ■ — 5-{h) = S^'+'ifo) = 0. 
Hence, fixing any element fo{Y) from K[Y]r, we determine all the coefficients 
/i, . . . ,/r from By Lemma |21 it is sufficient to show that the if [F] ''-module 
generated by d'^{K[YY) is finitely generated. By the theorem of Weitzenbock, 
-fir[y]* is a finitely generated algebra. Let {hi, . . . , be a generating set of 
Then d\K[Y]^) is spanned by the elements d^h^' ■ ■ ■ h^^). Since d is a derivation, 
d^{K\Y]^) is spanned by elements of the form 

hT--- Kr (n ('^i)) • • • (n C^")) , E *n + ■ • • + E = 

There is only a finite number of possibilities for , . . . , ti^ , and we obtain that 
d*{K\Y]^) generates a finitely generated ''-module. □ 

Corollary 4. Let, in the notation of this section, U and V be polynomial GL„i- 
modules, let g £ GLm be a unipotent matrix and let M (Y, Z) be equipped with the 
diagonal action of GLm- Then, for every GLm-submodule Mq of M{Y, Z), the natu- 
ral homomorphism M{Y, Z) — > M{Y, Z)/Mq induces an epimorphism M{Y, Z)^ — > 
{M (Y, Z) / Mq)^ , i. e. we can lift the g-invariants of M{Y,Z)/Mo to g-invariants 
ofM{Y,Z). 

Proof. The lifting of the constants was established in ^ in the case of relatively free 
algebras and the same proof works in our situation. Since U and V are polynomial 
GLm-modules, the module M{Y, Z) is completely reducible. Hence M{Y, Z) = 
Mo © M' for some GL„-submodule M' of M{Y,Z) and M{Y,Z)/Mo ^ M' . If 
w + Mo = w ^ {M{Y,Z)/Mo)3, then w = wq + w' , wo G Mq, w' G M', and 
g{w) — g{wo) + g{w'). Since g(u') = w, we obtain that g{w') ~ w' and the g- 
invariant w is lifted to the invariant w' . □ 

Remark 5. The proof of Proposition Ogives also an algorithm to find the genera- 
tors of M (y, Zy in terms of the generators of if [F]*. The latter problem is solved 
by van den Essen |Zj and his algorithm uses Grobner bases techniques. 

2. The Main Results 

The following theorem is the main result of our paper. For to = 2 it was estab- 
lished in 10 using the description of the g-invariants of K{x,y). 

Theorem 6. For any variety 5J of associative algebras which does not contain the 
algebra UT2{K) of 2x2 upper triangular matrices, the algebra of invariants Fm{VY 
of any unipotent g G GLm is finitely generated. 
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Proof. We shall work with the linear locally nilpotent derivation S = log 5 instead 
with g. 

It is well known that any variety QJ which does not contain UT2{K) satisfies 
some Engel identity [x2, xi, . . . , xi] = 0. By the theorem of Zelmanov |17| any Lie 
algebra over a field of characteristic zero satisfying the Engel identity is nilpotent. 
Hence we may assume that 2J satisfies the polynomial identity of Lie nilpotency 
[xi, . . . , Xc+i] = 0. (Actually, this follows from much easier and much earlier results 
on Pl-algebras.) 

Let us consider the vector space i?„i(5J) of so called proper polynomials in 
Fm{'^). It is spanned by all products [xi^ , . . . , a;^^.] • • • [xj-^ , . . . ,Xj^] of commuta- 
tors of length > 2. One of the main results of the paper by the author |3j states 
that if {/i, /2, . . .} is a basis of i3,„(QJ), then F„j(QJ) has a basis 



Let Sm be the homogeneous component of degree k of B,n{^). It follows 
from the proof of Theorem 5.5 in J^T, that for any Lie nilpotent variety 2J, and for 
a fixed positive integer m, the vector space i3m(2J) is finite dimensional. Hence 
Bm\^) = for n sufficiently large, e. g. for n > uq. Let Ik be the ideal of F,„(QJ) 
generated by (2J)+B^+^^ (QJ)-I- •■+S^''H2J). Since wxi = x,w+[w,x,], w e 

-Fm(5J), applying Lemma 2.4 4 , we obtain that Ik/Ik+i is a free left i4r[X]-module 
with any basis of the vector space Bm^ (53) as a set of free generators. Since (S is a 
nilpotent linear operator oiU = KX = Kxi © ■ • ■ © Kxm, it acts also as a nilpotent 
linear operator of Vk = Bm\^)- Proposition O gives that {Ik/Ik+iY is a finitely 
generated K[XY-modu\e. Clearly, B^'(2J) = K, B^rn' i^) = 0, Bin\^) is spanned 
by the commutators [xi^jXij], etc. Hence Iq/Ii = K[X] and by the theorem of 
Weitzenbock {Iq/IiY is a finitely generated algebra. We fix a system of generators 
/i, . . . , /a of the algebra {Iq/Ii)^ and finite sets of generators {fki, ■ ■ ■ , fkbk} of the 
is:[X] ■'-modules {h/Ik+if, k ^ 2,3, . . . ,nQ. The vector space [/ is a GLm-module 
and its GLm-action makes Vk a polynomial GLm-module. We apply CoroUary^land 
lift all fi and fkj to some (5-constants fi,fkj G Fm{^Y ■ The algebra S generated 
by /i, . . . , fa maps onto {Iq/IiY and hence [Ik/Ik+iY is a left S'-module generated 
by /fci, . • . , fkbk- The condition — together with Corollary 0] gives that the 



Together with the results of IB| Theorem gives immediately: 

Corollary 7. For m > 2 and for any fixed unipotent operator g G GL„i, g ^ 1, 
the algebra of g -invariants ^^(QJ)^ is finitely generated if and only if ^ does not 
contain the algebra UT2{K). 

We refer to the books P| and |S] for a background on the theory of matrix invari- 
ants. We fix an integer n > 1 and consider the generic n x n matrices xi, . . . , Xm- 
Let Cnm be the pure trace algebra, i. c. the algebra generated by the traces of 
products tr{xi-^ ■ ■ ■ Xi,,), k = 1,2,..., and let Tnm be the mixed trace algebra gener- 
ated hy xi, . . . , Xm and Cnm- It is well known that Cnm is finitely generated. (The 
Nagata-Higman theorem states that the nil polynomial identity = implies the 
identity of nilpotency xi ■ ■ ■ Xd = Q. If d{n) is the minimal d with this property, 
one may take as generators iT{xi-^ ■ ■ ■ xi^) with k < d{n).) Also, Tnm is a finitely 
generated G„r„-module. 



I > 0,^ = 1,2,...}. 



fi and fkj generate FmC^f ■ 



□ 
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Theorem 8. For any unipotent operator g £ GLm, the algebra is finitely 
generated. 

Proof. Consider the vector space U of all formal traces yi = tr(xij • • -a;;^), ij = 
1, ... ,771, 1 < k < d{n). Let Y be the set of all It has a natural structure 
of a GLm-module and hence 5 — logg acts as a nilpotent linear operator on U. 
Also, consider a finite system of generators /i,...,/a of the C„m-module Tnm- 
We may assume that the fj do not depend on the traces and fix some elements 
hj G K{X) such that hj fj under the natural homomorphism K{X) — > Tnm 
extending the mapping Xi ^ Xi, i = 1, . . . ,m. Let V be the GL„i-submodule of 
K{X) generated by the hj. Again, 5 acts as a nilpotent linear operator on V . We 
fix a basis Z = {zi, . . . , Zq] of V . Consider the free iir[y]-module M(F, Z) with 
basis Z. Proposition 131 gives that M{Y,ZY is a finitely generated if [F] ''-module 
and the theorem of Weitzenbock implies that /4r[F]'' is a finitely generated algebra. 
Since the algebra Cnm and the C„m-module Tnm are homomorphic images of the 
algebra K[Y] and the ii:[y]-module M{Y,Z), Corollary H gives that K\YY and 
M{Y,ZY map on C^,„ and T^^, respectively. Hence is a finitely generated 
module of the finitely generated algebra and, therefore, the algebra is 
finitely generated. □ 
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